Preparing for the visit

For those departments/schools that are to be visited (either because of an insufficiently high rating in a previous review or because of inclusion in a sample) much of the additional preparation for the review following the submission of the self-evaluation is concerned with the collation of necessary documentation.

In addition, steps need to be taken to ensure that all those involved in the review process are properly briefed and prepared for the visit.

Collecting the evidence

Subject review judgements are based on evidence it is your responsibility as the subject team to provide evidence that (a) your aims and learning outcomes are being achieved, and (b) supports your analysis of provision made in the self-evaluation.

The time and effort required to assemble the documentation for subject review should not be underestimated. You need to have started yesterday.

At the outset the department/school subject review coordinator should outline a plan for the collection of documents. For example, module/course leaders must be asked to provide the full range of information on modules/courses, including handouts, samples of student work, details of content, student questionnaires and teaching and learning strategies.

Note also that subject teams will need to ensure that sufficient secretarial/clerical resource is available to organise documentation as it is gathered together.

Historically, many institutions (especially those not from the CNAA tradition) have relied heavily on informal interactions between staff and with students for addressing issues relating to teaching and learning. However, in order to ensure that you have evidence to provide for subject reviewers you need to begin to document those procedures for which paperwork might not currently exist.

This will include:

• formally minuting key meetings, for example departmental/school committees, staff:student liaison meetings

• recording student attendance
• full recording of any annual course reviews
• responding in writing to external examiners’ comments and reports
• reporting to relevant committees the outcomes of action taken on the basis of student evaluations/comments/complaints (closing the loop)

• the development of files containing module/course outlines/handbooks, schedules of classes, handouts, assessment and examination details, student feedback and other evaluation of the module/course (if such files do not already exist, the department/school subject review coordinator will need to establish a template to aid collation).

Although evidence can be provided in other forms, including via discussion with subject reviewers, as much as possible should be documented.

Documentation for subject review

(The following section is an amended extract from the QAA
handbook.)

Apart from the self-evaluation, subject reviewers will not normally expect documents to be prepared especially for review. Departments/schools should direct reviewers, in the self-evaluation and by means of a separate list, to the availability and relevance of documents that might assist them to test and verify the statements made in the self-evaluation or which are relevant to the judgements they will make.

The following documents will be required in advance of the review:

• the self-evaluation, with the course specifications annexed
• relevant prospectuses
• a location map

The availability and relevance of further documentation will be discussed at the initial meeting with the department. As the review progresses, reviewers may ask for further documentation.

The following documents will be relevant to the review:

• department/school or course handbooks
• curricular documents such as module/course guides
• annual course monitoring or review reports, together with reports from external sources such as professional and/or statutory bodies, if these are available

• student questionnaire data
• external examiners’ reports for the previous three years
• student intake and progression data for the previous three years

The following documents may also be relevant, but this list is neither prescriptive nor exhaustive:

• minutes of relevant meetings, including examination boards
• equipment lists
• practical or placement handbooks
• course approval/validation and periodic course review documents
• further study and employment statistics (student destinations)
• academic staffing list and short profiles (indicating main teaching and research interests and any administrative responsibilities)

Reviewers will not necessarily ask for copies of documents. They may prefer to read the documents during the course of a visit. Documents can be provided in electronic form by mutual agreement between the subject provider and the review team.

According to the QAA there is no requirement or expectation that documents will be assembled in a ‘base room’ for the use of reviewers. If reviewers wish to see a document, they will ask for it.

Because review takes place over an extended period, immediate availability of every document that might be requested is not necessary. Nevertheless, you will wish be confident that you can easily locate and be certain of the validity of every document which reviewers might request.

Note that all material provided to reviewers should normally be passed via the subject review facilitator unless it is specifically agreed otherwise.

Student work

Reviewers will expect to see a sample of student work. The range and nature of student work to be made available to the reviewers will be discussed at the initial meeting.

Reviewers will look at student work to evaluate whether:

• student achievement matches the intended outcomes of the courses
• assessment is designed appropriately to measure achievement of the intended learning outcomes
• assessment instruments provide an adequate basis for discriminating between different categories of attainment
• the actual outcomes of programmes meet the minimum expectations for the award

Reviewers will not duplicate or ‘second-guess’ the work of external examiners. As such, reviewers will not normally expect to see work that is currently under consideration by external examiners.

Subject reviewers will need to see a broad sample of student work that demonstrates use of the full range of assessment instruments deployed in both formative and summative assessments.

To enable them to gain a full understanding of the assessment strategy, reviewers will need to see marking guides or other assessment criteria, and any guidance on providing feedback to students through assessment.

They will use external examiners’ reports to triangulate with their own observations of work from each level/year of study, samples of work from core modules and specialist options and from a representative range of attainment.

Samples of work may include, for example:

• coursework of various types
• projects and/or dissertations
• examination scripts

Departments/schools will need to consider well in advance of the review how best to collect and retain the necessary samples of student work, and should work on the basis of selecting three items in each class (ie first, upper second, lower second, third) for each piece of formative work, assessed work and each examination.

Marking and feedback sheets, and assessment criteria, should accompany the samples.
Where oral feedback has been given to students in addition to any written feedback then this should be clearly indicated.

It is the responsibility of departments/schools to check thoroughly the accuracy of marking and the consistency and quality of the feedback provided on the sample of work. This is a crucial task.

See also Appendix V for details of the issues which reviewers consider when reviewing student work.

Other items to include in the document list annex

Additional evidence to exemplify and support statements made in the self-evaluation might include:

Institution-level documents or central service information:

• the institution’s strategic plan
• your learning and teaching strategy
• the report of your most recent QAA institutional audit
• course regulations
• library information
• careers service information
• computing/IT services information
• students’ union information

Department/school items:

• details of student first destinations obtained from your careers service (including what happened to drop outs, their reasons for leaving etc). As a rough guide, details of at least the last three cohorts of graduates should be provided together with information on any particularly high achieving graduates from any era.
• lists (and profiles) of alumni who have done exceptionally well for themselves will add weight to claims about student achievement
• complete files of work done by a small sample of students, say four or five of differing abilities, during their entire academic career – this helps to illustrate progression

• list of publications by academic staff
• details of management and decision-making structures
• a list of addresses of relevant departmental/school websites
• employer testimonials
• charts showing the linkage between research output and relevant teaching, or which highlight key strengths of the subject
• charts offering a visual representation of course structures can be helpful for reviewers

Your central quality team should be able to provide more detailed guidance on the document list as it is being prepared.

Aside from collecting the necessary documentation, there are further activities that should be undertaken including:

• briefing support staff
• informing students about the visit and selecting and briefing those who will meet the reviewers
• contacting alumni and employers who are to be invited to meet the reviewers
• meetings and discussions with providers of central services (especially the careers service, the library and computing/IT services)
• a ‘dry run’

It is vital that everyone who may come into contact with the reviewers is briefed about the purpose of the subject review visit – the information provided to students is particularly important.

It is also important that departments/schools gain full benefit from previous experiences at their institution, especially the most recent visits and you should make direct contact with colleagues in subjects that have recently been reviewed.

The ‘dry run’

A ‘dry run’ can be an extremely valuable preparatory activity for subject review and can include:

• simulation of questioning by colleagues external to the department/school
• discussion of the self-evaluation, highlighting likely lines of enquiry and areas for further preparation
• feedback and action points
• exploration of issues likely to arise under each of areas covered in the self-evaluation document

You will wish to involve colleagues external to the department/school and with experience of subject review in any dry run in order to make the exercise as valuable as possible.

Observation of teaching

Departments/schools are strongly advised to take the opportunity presented by subject review to develop a peer observation scheme if one does not already operate in the department.

Of all the activities which can be undertaken in preparing for subject review, peer observation may prove to be the most valuable in contributing to the long term enhancement of the quality of teaching and learning. Moreover, it makes it much less likely that the subject review itself will involve any observation of teaching, thereby offering the immediate prospect of a genuinely lighter touch.

In the event that observation of teaching does occur, a set of protocols (see Appendix VI) will be operated.

Other preparatory activities

Beyond dry runs and document gathering there are some other preparations for the visit which need to be made (although some of these may not be feasible until after the initial meeting with the reviewers):

• finalising the timetable for meetings with key individuals within the department/school and relevant central services personnel (noting that the review may be over an extended period)
• ensuring that the teaching environment and common areas are well-presented and look cared for (for example check all noticeboards for old and tatty posters)
• making any necessary domestic arrangements, for example booking lunches for student meeting, ensuring visiting alumni have accommodation booked if needed
• final briefing of all staff
• dispatch of additional documentation to the subject review team

Last Modified: 30 June 2010